41 research outputs found
Maximum Entropy Heterogeneous-Agent Mirror Learning
Multi-agent reinforcement learning (MARL) has been shown effective for
cooperative games in recent years. However, existing state-of-the-art methods
face challenges related to sample inefficiency, brittleness regarding
hyperparameters, and the risk of converging to a suboptimal Nash Equilibrium.
To resolve these issues, in this paper, we propose a novel theoretical
framework, named Maximum Entropy Heterogeneous-Agent Mirror Learning (MEHAML),
that leverages the maximum entropy principle to design maximum entropy MARL
actor-critic algorithms. We prove that algorithms derived from the MEHAML
framework enjoy the desired properties of the monotonic improvement of the
joint maximum entropy objective and the convergence to quantal response
equilibrium (QRE). The practicality of MEHAML is demonstrated by developing a
MEHAML extension of the widely used RL algorithm, HASAC (for soft
actor-critic), which shows significant improvements in exploration and
robustness on three challenging benchmarks: Multi-Agent MuJoCo, StarCraftII,
and Google Research Football. Our results show that HASAC outperforms strong
baseline methods such as HATD3, HAPPO, QMIX, and MAPPO, thereby establishing
the new state of the art. See our project page at
https://sites.google.com/view/mehaml
Hawkeye: Change-targeted Testing for Android Apps based on Deep Reinforcement Learning
Android Apps are frequently updated to keep up with changing user, hardware,
and business demands. Ensuring the correctness of App updates through extensive
testing is crucial to avoid potential bugs reaching the end user. Existing
Android testing tools generate GUI events focussing on improving the test
coverage of the entire App rather than prioritising updates and its impacted
elements. Recent research has proposed change-focused testing but relies on
random exploration to exercise the updates and impacted GUI elements that is
ineffective and slow for large complex Apps with a huge input exploration
space. We propose directed testing of App updates with Hawkeye that is able to
prioritise executing GUI actions associated with code changes based on deep
reinforcement learning from historical exploration data. Our empirical
evaluation compares Hawkeye with state-of-the-art model-based and reinforcement
learning-based testing tools FastBot2 and ARES using 10 popular open-source and
1 commercial App. We find that Hawkeye is able to generate GUI event sequences
targeting changed functions more reliably than FastBot2 and ARES for the open
source Apps and the large commercial App. Hawkeye achieves comparable
performance on smaller open source Apps with a more tractable exploration
space. The industrial deployment of Hawkeye in the development pipeline also
shows that Hawkeye is ideal to perform smoke testing for merge requests of a
complicated commercial App
ImMesh: An Immediate LiDAR Localization and Meshing Framework
In this paper, we propose a novel LiDAR(-inertial) odometry and mapping
framework to achieve the goal of simultaneous localization and meshing in
real-time. This proposed framework termed ImMesh comprises four tightly-coupled
modules: receiver, localization, meshing, and broadcaster. The localization
module utilizes the prepossessed sensor data from the receiver, estimates the
sensor pose online by registering LiDAR scans to maps, and dynamically grows
the map. Then, our meshing module takes the registered LiDAR scan for
incrementally reconstructing the triangle mesh on the fly. Finally, the
real-time odometry, map, and mesh are published via our broadcaster. The key
contribution of this work is the meshing module, which represents a scene by an
efficient hierarchical voxels structure, performs fast finding of voxels
observed by new scans, and reconstructs triangle facets in each voxel in an
incremental manner. This voxel-wise meshing operation is delicately designed
for the purpose of efficiency; it first performs a dimension reduction by
projecting 3D points to a 2D local plane contained in the voxel, and then
executes the meshing operation with pull, commit and push steps for incremental
reconstruction of triangle facets. To the best of our knowledge, this is the
first work in literature that can reconstruct online the triangle mesh of
large-scale scenes, just relying on a standard CPU without GPU acceleration. To
share our findings and make contributions to the community, we make our code
publicly available on our GitHub: https://github.com/hku-mars/ImMesh
Application of Zebrafish Models in Inflammatory Bowel Disease
Inflammatory bowel disease (IBD) is a chronic, recurrent, and remitting inflammatory disease with unclear etiology. As a clinically frequent disease, it can affect individuals throughout their lives, with multiple complications. Unfortunately, traditional murine models are not efficient for the further study of IBD. Thus, effective and convenient animal models are needed. Zebrafish have been used as model organisms to investigate IBD because of their suggested highly genetic similarity to humans and their superiority as laboratory models. The zebrafish model has been used to study the composition of intestinal microbiota, novel genes, and therapeutic approaches. The pathogenesis of IBD is still unclear and many risk factors remain unidentified. In this review, we compare traditional murine models and zebrafish models in terms of advantages, pathogenesis, and drug discovery screening for IBD. We also review the progress and deficiencies of the zebrafish model for scientific applications